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ABSTRACT 

A pilot version of a School Capacity Assessment (SCA) was 
developed in 2002 to assess the degree to which schools possess the potential 
to become high performing learning communities. The SCA was part of AEL' s 
School Capacity Development project. The pilot version of the SCA was 
intended to be administered to K-12 professional staff to assist them in 
determining how well positioned their schools are to begin the development of 
a higher performing learning community. The 99-item, 4-page instrument 
contains 8 subscales: Collective Teacher Efficacy, Deprivatized Practice, 
Program Coherence, Technical Resources, Equitable Practice, Differentiated 
Instruction, Expectations for Student Performance, and Time for Planning. The 
pilot SCA was completed by 427 respondents from 10 schools in a Virginia 
school district and 26 respondents from a West Virginia school district. 
Results suggest that the SCA holds potential for assessing school capacity 
for improvement . As was to be expected from the sample of relatively low 
performing schools chosen, item and subscale means were relatively low. The 
instrument overall was internally consistent in the pilot test, and 
correlations between the subscales were, in general, moderate to very strong. 
Recommendations are made for additional study. Three appendixes contain the 
SCA, descriptive statistics, and a checklist for applying standards. 

(Contains 13 tables and 76 references.) (SLD) 
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EXECUTIVE SUMMARY 



A pilot version of AEL’s School Capacity Assessment (SCA) was developed in the 
spring of 2002 by Caitlin Howley and Joy Riffle to assess the degree to which schools possess 
the potential to become high performing learning communities. This research and development 
focus grows from the Department of Education’s Office of Educational Research and 
Improvement’s concern with and commitment to investigating how low-performing schools may 
be transformed into learning communities for students, faculty, and community members. More 
specifically, the SCA was developed in response to AEL’s School Capacity Development 
project, staff of which required an instrument to assess their efforts to enhance the capacity to 
improve in partner schools. 

Based on a review of the education research on change, school capacity was defined by 
AEL research and evaluation staff as the presence of certain school structures, and teacher 
practices and perspectives, needed to support the growth of a thriving learning community. 
School cultural and attitudinal factors were incorporated in this view of school capacity for 
improvement (Kruse, Louis, & Bryk, 1995). But structural components were also included in 
response to research showing the importance of school structures and policies to successful 
improvement initiatives (e.g., Fullan, 1991, 1994; Hord, Rutherford, Huling- Austin, & Hall, 
1987; Howley & Brown, 2001; Kruse, Louis, & Bryk, 1995; Newmann, King, & Youngs, 2001). 
It is hypothesized that, lacking these structures, practices, and perspectives, school staff will be 
less likely to nurture and sustain significant school improvement. 

AEL’s pilot version of the SCA was developed in response to the paucity of definition, 
operationalization, and assessment of school capacity in the education research and evaluation 
literature. It is intended for administration to K-12 school professional staff. Data from 
administration of the survey are to assist school staff in ascertaining how well positioned their 
schools are to begin the development of a high performing learning community. In addition, 
subscale data will allow staff to identify dimensions of school capacity in need of further 
development in their schools. The instrument is intended for diagnostic use, for instance at the 
beginning of school reform efforts. It is also intended for administration and analysis over the 
course of school improvement undertakings. 

The SCA is a 99-item, four-page instrument (see Appendix A). Response options to the 
items are forced-choice, using a scale of 1 to 4, in which 1 means “strongly disagree,” 2 means 
“disagree,” 3 means “agree,” and 4 means “strongly agree.” Subscale items are randomly 
distributed throughout the instrument so that subscales are not readily apparent to respondents. 
The instrument is available in a machine scannable format. 

Eight subscales constitute the survey: Collective Teacher Efficacy, Deprivatized Practice, 
Program Coherence, Technical Resources, Equitable Practice, Differentiated Instruction, 
Expectations for Student Performance, and Time for Planning. All eight subscales are either 
drawn directly from other research endeavors or are the result of syntheses of research efforts 
that did not necessarily produce assessment instruments. 
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The purpose of the pilot test of AEL’s SCA was to begin an exploration of the 
psychometric properties of the instrument and its subscales. AEL staff wanted to discover the 
correlations between items intended to constitute distinct subscales and assess discrete concepts, 
and to delete items not highly correlated with others in their respective subscales. In other words, 
AEL staff sought data reduction, as the 99-item instrument is cumbersome. Staff also were 
interested in the degree to which subscales were reliable. In short, an exploratory analysis of the 
SCA’s statistical properties was wanted. 

The pilot version of AEL’s SCA was administered to school staff in two school districts 
in the summer of 2002. Both districts have histories of social, economic, and political 
difficulties, and schools in both have had several decades of depressed student achievement. 
District administrative staff agreed that AEL would administer the SCA as part of its research 
and development efforts there, and that data from the initial administration would serve as 
baseline data for a longitudinal study of the effects of AEL’s work. 

Five hundred copies of the instrument were mailed to the superintendent of a southern 
Virginia school division (i.e., district) collaborating with AEL in the early summer. Surveys from 
10 schools in the division and a total of 427 respondents were returned to AEL by mid-July 
2002. 



In addition, the SCA was administered to teams from a West Virginia school district 
attending an AEL summer institute on project-based learning in July 2002. Approximately 40 
copies of the instrument were distributed by an AEL evaluator to teams from 10 schools at the 
close of the institute’s first day. Participants were asked to complete the SCA that evening and 
return it to AEL staff the following morning. Twenty-six respondents completed and returned the 



SCA. 



Based on the results of the pilot test, it was concluded that the SCA appears to hold some 
promise for assessing school capacity for improvement. As would be expected given the nature 
of the sample of low-performing schools, item and subscale means were relatively low. In 
addition, the sample evidenced a negative skew on nearly all of the subscales. These findings 
suggest that the SCA is able to identify struggling schools, although it is yet to be seen if the 
instrument is also capable of identifying schools that possess a great degree of capacity for 
improvement. 

In terms of reliability, the instrument overall was internally consistent in the pilot test 
administrations. In addition, most of the sub scales possess sufficient internal consistency 
reliability. Correlations and exploratory factor analyses indicated the relationship of subscale 
items to one another. Most subscale correlation matrices confirmed the moderate to strong 
relationship between subscale items. Exploratory factor analyses differentiated further the 
Equitable Practice subscale into the Anti-Discriminatory Teaching and Responsive Pedagogy 
subscales. Items in each are moderately to highly correlated. 

Correlations between the subscales were, in general, moderate to very strong. Subscales 
assessing structural conditions were highly correlated with one another, whereas the subscales 
evaluating practice and attitudinal stances were very highly to moderately correlated. This 
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suggests that the overall instrument effectively assesses the two sorts of subscales, and that the 
subscales are interrelated without sacrificing their distinct measures. 

It was recommended that AEL staff further develop the SCA. For instance, items not 
correlating with other subscale items should be deleted. The Equitable Practice subscale should 
also be divided into the two subscales discerned by the exploratory factor analysis. 

It was also suggested that staff conduct a field test, in which a larger, more differentiated 
sample would be used. Ideally, the SCA should be administered to an equal number of 
respondents from low-, moderate-, and high-performing schools so as to test its ability to 
discriminate between the various levels of school performance. A larger, more diverse sample 
would also allow factor analysis for the purpose of more rigorously and empirically defining the 
subscales. 
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INTRODUCTION 



Since the 1960’s, American schools have been under especial scrutiny for their capacity 
to educate youth effectively. Although school reform and improvement have always been 
national concerns (the Progressive era at the turn of the last century, for example), the launching 
of Sputnik in 1957, at a time when the Cold War shaped American fears, spurred alarm about the 
state of schooling in the country. If the Russians, who appeared to live under less prosperous 
conditions, were capable of such a scientific feat, citizens wondered, why had Americans not 
launched the first orbital satellite? One of the most frequently-cited answers was that United 
States schools were not educating students sufficiently, particularly in subject areas of increasing 
prominence, such as math and science. The launch of Sputnik proved pivotal in our ongoing and 
contemporary concern with school improvement. 

A number of school improvement trends have arisen since the 1960s in attempts to 
improve American education, each offering particular antidotes to educational troubles. 
Decentralization efforts in the 1960s and 1970s were approaches that sought to encourage local 
control of curriculum and finance, and to increase community participation in matters of 
education. Ultimately, however, many of these efforts became ineffective in terms of school 
improvement as involvement of community members was often token, or dominated only by the 
most influential community leaders (deMarrais & LeCompte, 1999). 

In the 1990s, site-based management and shared decision-making were successors to the 
earlier decentralization efforts. These school improvement approaches sought again to render 
schools more responsive to community concerns. Nonetheless, participants with relatively little 
power continued to face obstacles to their full involvement, and research revealed little impact of 
site-based management or shared decision making on academic indicators (deMarrais & 
LeCompte, 1999; Riordan, 1997). 

Another wave of school improvement efforts, in response to the 1983 National 
Commission on Excellence in Education’s report A nation at risk: The imperative for 
educational reform, focused on raising standards for students and teachers. This approach 
entailed establishing performance requirements for students and linking teacher accountability to 
student achievement on standardized tests. The standards movement continues to play a 
significant role in contemporary debate about how to improve education (Riordan, 1997). 

The Effective Schools movement was an attempt to discover what might make some 
schools better equipped than others to produce high performing students. According to this 
research (Levine & Lezotte, 1995), effective schools evidence specific characteristics, such as a 
clear mission, high academic expectations for all students, a safe school environment, and strong 
instructional leadership from administrators. However, this area of research failed to provide 
insight into how schools developed such characteristics. 

School improvement is increasingly viewed as an ongoing and comprehensive process. 
Recent legislation has encouraged the adoption of such a view, with the 1998 appropriation of 
$150 million by Congress to states for allocation to schools undertaking research-based 
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schoolwide reform programs through the Comprehensive School Reform Demonstration 
Program (CSRD). Earlier, in 1994, Congress altered regulations to allow schools receiving 
Title I funds, with free and reduced lunch 50% and above, to use such funds for whole school 
improvement (American Institutes for Research, 1999). 

The reform models mentioned in the legislation instituting CSRD encompass a variety of 
approaches to reform, from skill-based, to comprehensive, to processual. In addition, the models 
vary in their degree of prescriptiveness. All claim to be based upon research and to have 
evidence of some positive impact. Yet investigations of and prototypes for school improvement 
extend far beyond the models forwarded in CSRD legislation: Contemporary literature on school 
improvement has roots in the school effectiveness literature of the 1970s and early 80s 
mentioned earlier (e.g., Levine & Lezotte, 1995). 

Much current prescriptive education literature and some research suggests that the 
interplay between school cultural and structural conditions significantly affects how change at a 
particular school will be greeted (e.g., Newmann & Wehlage, 1996). They contend that if 
cultural characteristics, such as commitment to high expectations, support for inquiry, and caring 
relationships, intersect with structural factors, such as time for staff development and freedom 
from excessive organizational constraints, school reform will proceed more smoothly. These 
structural and cultural conditions can be seen as contributing to school capacity for improvement 
(Newmann, King, & Youngs, 2001). 

Along with these intersections, school leadership must be an integral part of improvement 
efforts (van der Bogert, 1998), and collaboration among the many stakeholders in school 
communities must be pursued (Sarason & Lorentz, 1998). Fullan and Miles (1994) additionally 
suggest that those involved in improvement must recognize that it is a process, filled with 
ambiguity, uncertainty, and risk, rather than a scripted, easily implemented recipe. Moreover, 
Fullan’s most important insight is that school reform will not proceed without the voluntary 
support of staff who view the reform as meaningful and in alignment with their own worldviews 
(Fullan, 1991). 

Thus, efforts to improve schools are an ongoing and contemporary national concern. 
Research and policy in education are often devoted to imagining, mandating, defending, 
resisting, and assessing a wide variety of improvement strategies. Nonetheless, the majority of 
reforms have not resulted in significant change in practice (Cuban, 1993) or in student 
performance (American Institutes for Research, 1999; deMarrais & LeCompte, 1999; Riordan, 
1997). As Brown, Halsey Lauder, and Wells (1997) imply, and as Anyon (1997) vividly 
demonstrates, other contextual factors play a pivotal role in how, and whether, school change is 
enacted. Newmann, King, and Youngs (2001) likewise suggest that school reform efforts interact 
with their context, part of which is school capacity for improvement. It is this important notion of 
school capacity that is the subject of the following section. 

AEL’s School Capacity Assessment — Pilot Version 

A pilot version of AEL’s School Capacity Assessment (SCA) was developed in the 
spring of 2002 by Caitlin Howley and Joy Riffle to assess the degree to which schools possess 
the potential to become high performing learning communities. This research and development 
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focus grows from the Department of Education’s Office of Educational Research and 
Improvement’s concern with and commitment to investigating how low-performing schools may 
be transformed into learning communities for students, faculty, and community members. More 
specifically, the SCA was developed in response to AEL’s School Capacity Development 
project, staff of which required an instrument to assess their efforts to enhance the capacity to 
improve in partner schools. 

Based on a review of the education research on change, school capacity was defined by 
AEL research and evaluation staff as the presence of certain school structures, and teacher 
practices and perspectives, needed to support the growth of a thriving learning community. 
School cultural and attitudinal factors were incorporated in this view of school capacity for 
improvement (Kruse, Louis, & Bryk, 1995). But structural components were also included in 
response to research showing the importance of school structures and policies to successful 
improvement initiatives (e.g., Fullan, 1991, 1994; Hord, Rutherford, Huling-Austin, & Hall, 
1987; Howley & Brown, 2001; Kruse, Louis, & Bryk, 1995; Newmann, King, & Youngs, 2001). 
It is hypothesized that, lacking these structures, practices, and perspectives, school staff will be 
less likely to nurture and sustain significant school improvement. 

Newmann and his colleagues (2001) contend that structural conditions, such as program 
coherence and alignment, the sufficiency of technical and professional resources, and the 
provision of adequate time for staff to plan collaboratively and/or implement change, are critical 
to the likelihood that school reform will be undertaken with commitment. Moreover, school 
improvement efforts cannot be sustained over time without sufficient support from district and 
school policies and structures (Howley & Brown, 2001). Structural conditions, though often 
invisible or taken for granted, significantly shape how people behave, of what they believe they 
(and their students) are capable, and to what they commit themselves (Bourdieu & Passeron, 
1997; deMarrais & LeCompte; Fullan, 1991; Mills, 1959; Riordan, 1997). 

In addition, teachers’ practice also plays an important role in forecasting the success of 
school reform efforts. Louis, Marks, and Kruse (1996) illustrate how deprivatized practice, in 
which school staff regularly observe one another and provide constructive feedback, structures a 
conduit by which other change efforts may be brought to fruition. Meaningful collaboration 
becomes possible when staff are in the habit of crossing the thresholds of each other’s classroom 
doors. 



Equitable teaching practices and differentiated instruction together constitute a nuanced 
pedagogy that is at once attentive, equitable, and sensitive. As Darling-Hammond notes, 
“Successful education can occur only if teachers are prepared to meet rigorous learning demands 
and the different needs of students” (1997, p. 334). Teachers who are accustomed to applying 
themselves equitably to diverse students are better equipped to confront the challenges wrought 
by social, economic, and political devastation in low-performing schools and their communities 
(Anyon, 1997; Paley, 1979). However, it could also be argued that school staff are more likely to 
undertake serious change with commitment if they are already in the practice of differentiating 
instruction in ways intended to support their students fully and adequately. 
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Teachers’ attitudes, perceptions, expectations, and assessments are also closely bound to 
the likelihood that their school is well-positioned to undertake significant school improvement 
work. Faculty who believe that they are not capable as a group of teaching their students are not 
likely to have much faith in their attempts to effect any broader change in their school. Collective 
teacher efficacy is critical to the capacity schools possess for committing to and implementing 
improvement efforts (Goddard, Hoy, & Hoy, 2000). 

Expectations for student performance, as with teacher efficacy, constitute an important 
gauge of school capacity. Depressed expectations indicate a professional fatalism not conducive 
to improvement or, obviously, enhanced student achievement (Tauber, 1998). In addition, 
schools with capacity are schools with a predisposition toward nurturing learning. If teachers do 
not expect much from their students, their school cannot possess much capacity for nurturing 
student achievement. 

AEL’s pilot version of the SCA was developed in response to the paucity of definition, 
operationalization, and assessment of school capacity in the education research and evaluation 
literature. It is intended for administration to K-12 school professional staff. Data from 
administration of the survey are to assist school staff in ascertaining how well positioned their 
schools are to begin the development of a high performing learning community. In addition, 
subscale data will allow staff to identify dimensions of school capacity in need of further 
development in their schools. The instrument is intended for diagnostic use, for instance at the 
beginning of school reform efforts. It is also intended for administration and analysis over the 
course of school improvement undertakings. 

The SCA is a 99-item, four-page instrument (see Appendix A). Response options to the 
items are forced-choice, using a scale of 1 to 4, in which 1 means “strongly disagree,” 2 means 
“disagree,” 3 means “agree,” and 4 means “strongly agree.” Subscale items are randomly 
distributed throughout the instrument so that subscales are not readily apparent to respondents. 
The instrument is available in a machine scannable format. 

Eight subscales constitute the survey: Collective Teacher Efficacy, Deprivatized Practice, 
Program Coherence, Technical Resources, Equitable Practice, Differentiated Instruction, 
Expectations for Student Performance, and Time for Planning. All eight subscales are either 
drawn directly from other research endeavors or are the result of syntheses of research efforts 
that did not necessarily produce assessment instruments. 

The first two subscales have been previously validated. They are defined as follows: 

• Collective Teacher Efficacy: a 12-item scale assessing “the extent to which a faculty 
believes in its conjoint capability to positively influence student learning” (Goddard, 
2002, p. 97) 

• Deprivatized Practice: a 7-item scale assessing “the frequency with which teachers 
observe each other’s classes to critique colleagues’ teaching and provide meaningful 
feedback; it also measures the frequency of constructive reviews from supervisors” 
(Louis, Marks, & Kruse, 1996, p. 769) 
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The remaining subscales are currently being pilot tested in an effort to establish their validity and 
reliability. These scales are defined as follows: 

• Program Coherence: a 12-item scale measuring “the extent to which the school’s 
programs for student and staff learning are coordinated, focused on clear learning goals, 
and sustained over a period of time” (Newmann, King, & Youngs, 2001, p. 6) 

• Technical Resources: a 7-item scale evaluating the availability to faculty of working 
equipment, technology, instructional materials, facilities, and professional resource 
materials, such as journals (Newmann, King, & Youngs, 2001) 

• Equitable Practice: a 38-item scale measuring the degree to which faculty understand 
diversity and engage in classroom practices that equitably support the learning of all 
students (deMarrais & LeCompte, 1999; Pohan & Aguilar, 2001; Sadker & Sadker, 1994; 
University of Minnesota, Diversity Work Group, 2002) 

• Differentiated Instruction: an 8-item scale assessing the extent to which faculty adapt 
their instructional strategies and grouping arrangements to meet the learning needs of 
diverse students (Baber, C.R., 2001; Tomlinson, 1995, 1999a-b, 2000; University of 
North Carolina, 2001) 

• Expectations for Student Performance: a 10-item scale evaluating the degree to which 
faculty believe their students are capable of mastering material presented to them and the 
level at which teachers anticipate that their students will perform (Baber, 2001 ; Bourdieu 
& Passeron, 1997; deMarrais & LeCompte, 1999; McLeod, 1987; Ogbu, 1983; Paley, 
1979; Riordan, 1997; University of North Carolina, 2001; Willis, 1981) 

• Time for Planning: a 5 -item scale assessing the extent to which school staff have 
sufficient dedicated time for planning and teaching (Abdal-Haqq, 1996; Lashway, 1998). 

The importance of each subscale to a conceptualization of school capacity is explained 
below. It should be noted that three subscales are intended to assess various structural conditions 
under which teachers work; these are the Program Coherence, Technical Resources, and Time 
for Planning measures. The Deprivatized Practice, Equitable Practice, and Differentiated 
Instruction subscales are meant to ascertain teacher practices, and the Expectations for Student 
Performance subscale is primarily attitudinal. 

Collective Teacher Efficacy 

Collective teacher efficacy extends the notion of individual teacher efficacy to a faculty’s 
shared sense of capacity to effect positive student outcomes. Whereas an individual’s assessment 
of his or her own efficacy as a teacher may vary according to specific contexts (such as class 
size, subject area, or student demographics), a measure of collective teacher efficacy provides a 
more global evaluation of the specific social and organizational context in which a faculty works. 
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Teachers’ shared beliefs about their collective ability to teach students effectively is, according 
to Goddard, Hoy, and Hoy (2000), a better gauge of school capacity than measures of individual 
efficacy or internal locus of control. Collective teacher efficacy is “an emergent group-level 
attribute, the product of the interactive dynamics of the group members. As such, this emergent 
property is more than the sum of the individual attributes” (p. 482). 

Further, collective teacher efficacy is “a way of conceptualizing the normative 
environment of a school and its influence on both personal and organizational behavior” 
(Goddard, 1998, p.65). Teachers’ perceptions of their faculty’s ability to teach with efficacy 
shapes their strivings and behaviors in the classroom. Thus, if teachers believe themselves to 
belong to a very efficacious faculty, “the normative environment will press teachers to persist in 
their educational efforts (Goddard, 1998, p. 65). On the other hand, a faculty with little sense of 
collective efficacy will be less likely to exert normative pressure on each other to undertake 
rigorous pedagogy. 

Because of its link to faculty behavior and its hypothesized (Goddard, 1998, 2002; 
Goddard, Hoy, & Hoy, 2000) and tentatively confirmed (Goddard, Hoy, & Hoy, 2002) impact on 
student achievement, collective teacher efficacy appears to constitute an important component of 
school capacity for improvement. A faculty which does not believe in its capabilities will not 
likely impel itself toward improvement. However, a faculty with a strong sense of its ability to 
effect change in student achievement will be better positioned to seek improvement. 

Goddard’s (2002) revision of his earlier measure of collective teacher efficacy was 
adopted for inclusion in AEL’s pilot version of the SCA. The 12-item revision possesses 
adequate internal consistency reliability with a Cronbach’s alpha coefficient of .94. Moreover, 
Goddard’s analysis indicates that the new version is valid; the revised measure correlates highly 
with the earlier instrument, and multilevel tests of predictive validity showed that the new 
version is a good predictor of between-school variability in student mathematics achievement. 

Deprivatized Practice 

Louis, Marks, and Kruse (1996) contend that, among other phenomena, deprivatized 
practice is pivotal in the development of school professional community. In this view, 
deprivatized practice is the degree to which faculty observe one another’s work, provide 
feedback, and serve as mutual mentors or coaches. Schools in which practice is deprivatized tend 
to view teaching less as an autonomous individual project and more as a collaborative 
undertaking (Sarason & Lorentz, 1998). As a result, faculty in such schools experience less 
professional isolation and greater opportunity for learning from colleagues (Education 
Commission of the States, 1996). Deprivatized practice, then, provides faculty with a wider 
network of resources. 

In terms of school capacity for improvement, serious change is not likely to take hold if 
faculty are not aided by norms or mechanisms that support collegial learning, critique, and cross- 
fertilization. As Cuban’s (1993) historical analysis of school change reveals, professional 
isolation and conservative norms in schools have rendered most improvement efforts irrelevant, 
and ultimately teachers have made very few serious changes in their practice as a result. 
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However, schools that provide the structural support for deprivatized practice invite 
collaboration and collegiality, which in turn invite opportunities for sustainable improvement 
(Corallo & McDonald, 2002). 

The 7-item Deprivatized Practice subscale is a closed-response option adaptation by 
Meehan and Cowley (1998) to the original open-ended questionnaire developed by Louis, 

Marks, and Kruse (1996). Although the 1998 administration of the adaptation by Meehan and 
Cowley indicated that the subscale possessed less than ideal reliability, with Cronbach’s alphas 
ranging between .65 to .69, a later administration by Nilsen revealed the scale to be more 
reliable, with an alpha of .84. 

Program Coherence 

An important structural condition supporting school capacity for improvement is 
instructional program coherence. According to Newmann, King, and Youngs (2001), program 
coherence is a measure of the extent to which a school is sufficiently programmatically 
integrated. The continual and shifting presence of unrelated, unfocused, and multiple 
improvement programs weakens schools’ organizational efficacy. Aligned initiatives, 
implemented and monitored carefully for sustained periods of time, on the other hand, at the very 
minimum do not detract from a school’s efforts to educate students. 

Program coherence also encompasses the alignment of curriculum and instruction within 
grade levels and between grade levels (Corallo & McDonald, 2002; Newmann, Smith, 
Allensworth, & Bryk, 2001). Adequate alignment and sequencing assists in the maintenance of 
an appropriate intellectual pace and rigor, and focuses attention on the primary purpose of 
education. It also reduces redundancy and fosters communication and collaboration among 
teachers. 

Program coherence is viewed as critical to school capacity for improvement because 
schools struggling to implement many unrelated programs are not immediately equipped to 
undertake significant improvement work. Already burdened with other competing and shifting 
priorities, teachers in schools with little programmatic coherence are unlikely to accommodate 
additional serious change. Focus and carefully allocated resources to a committed, shared 
purpose prepare a more hospitable environment for improvement. 

The Program Coherence subscale on AEL’s SCA is an adaptation of items from a survey 
of professional development to build school capacity. In addition, several other items were added 
by AEL staff. No reliability or validity analyses were provided by Newmann, King, and Youngs, 
although their study seems to confirm that program coherence constitutes a critical component of 
school capacity for improvement. 

Technical Resources 

Newmann, King, and Youngs (2001) also found the presence of adequate technical and 
professional resources to be a useful indicator of school capacity for improvement. Instructional 
materials, functioning technical and computer equipment, and adequate workspace represent 
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some of the material conditions under which teachers work. Improvement efforts which depend 
on such tools are likely to fail if teachers do not have access to them. 

In addition, teachers who feel that they do not have the material resources with which to 
teach to their objectives in the classroom will feel additionally hampered if asked to institute 
significant change across their school. If teachers’ fundamental resource needs are unmet, the 
likelihood that their school can effect and sustain improvement is small. 

As with the Program Coherence subscale, the Technical Resources subscale is an 
adaptation of survey items developed by Newmann, King, and Youngs (2001). Some items were 
used verbatim, others were modified, and still others were developed by AEL staff to extend and 
elaborate on the concept assessed by the subscale. Reliability and validity information about the 
items is not available. 

Equitable Practice 

Schools are increasingly diverse organizations, with larger percentages of African 
American and Latino/a students. In addition, national attention is focused on increasing the 
academic achievement of racially/ethnically-defined youth and of low SES students (Fortune, 
2002; Schwartz, 2001a). Education Week, for example, covered the issue in 2000 with a four- 
part series (Johnston & Viadero, 2000; Viadero, 2000; Viadero & Johnston, 2000a, 2000b). 
Equitable education for all students is, however, both a national challenge and a legal imperative 
since the 1954 Brown v. Board of Education Supreme Court decision, which overturned the 
"separate but equal" doctrine justifying school segregation by racial category. 

Equity must also be applied to gender, as much research indicates that curriculum and 
instruction tend to favor boys (deMarrais & LeCompte, 1999; Sadker & Sadker, 1994). For 
instance, boys may receive more attention, praise, and opportunities to elaborate or correct their 
answers to instructional questions (Mid- Atlantic Equity Center, 1993). Female figures appear 
less often in literary or historical accounts in curricula, and girls confront sexist language at 
school in which being called female is an insult (Thome, 1995). In addition, girls enroll in fewer 
advanced math and science courses than do their male counterparts (Perez, 2000). 

Equitable practice can be defined in numerous ways, along multiple dimensions. Rose 
(1999), for instance, identifies 10 indicators of fair teaching, ranging from equal distribution of 
response opportunities to courtesy and respect. The University of North Carolina Diversity Work 
Group (2002) cites a long list of practices identified by educators as conducive to the 
development of an equitable environment. Kahle (2002) explicates a variety of strategies to 
enhance the equity of science teaching, and Rickford (2001) illustrates how the use of culturally 
relevant texts and higher order questioning techniques are useful strategies for engaging low SES 
and ethnic minority students. Ensuring that curriculum and discipline practices honor students’ 
backgrounds is another strategy suggested as important to creating an equitable classroom 
(Thompson & O’Quinn, 2001). Multicultural education research also points up a wealth of 
practices that ensure students receive equitable educational opportunities (c.f, Banks & Banks, 
1995). Ultimately, equitable practice is a multiple concept: More than one strategy is required 
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for the creation and sustenance of an academic environment that is fair and sensitive to all 
students (NWREL, 1997). 

Schools equipped to teach their students equitably, fairly, yet also sensitively are likewise 
equipped to make improvement equitably. Improvement can hardly be considered full and 
meaningful unless it is salient to the experience and achievement of all students. 

The Equitable Practice subscale of AEL’s pilot version of the SCA was developed by 
AEL staff using the research literature cited above as a catalyst. Items were constructed to 
account for a variety of equitable practices, including racially/ethnically and socioeconomically 
sensitive pedagogy, relevant curriculum, active discouragement of stereotypical comments and 
behavior, equitable praise, multicultural content, and use of students’ preferred speaking styles to 
enhance learning. 

Differentiated Instruction 

Classrooms are not homogenously populated; students hail from various communities, 
bring disparate skills and strengths, and have differing academic needs. Varying content, process, 
products, and learning environment to meet students’ assorted needs is differentiating instruction 
(Tomlinson, 2000). The University of North Carolina’s School of Education (2001) makes the 
teaching of differentiated instructional strategies to pre-service teachers one of its priorities 
because it is considered so essential to effective pedagogy. 

The rationales for differentiating instruction are many. Instruction that honors the 
linguistic and literacy styles of young children augments their reading skills (Vemon-Feagans, 
Hammer, Miccio, & Manlove, 2001), and by extension, their learning of any subject that requires 
literacy skills. Moreover, differentiated instruction has been shown to improve student 
achievement (Dahl, Scharer, Lawson, & Grogan, 1999; although see Rowan & Miracle, 1983, 
for an alternative view). Differentiated instruction accommodates students of various cognitive 
abilities (Tomlinson, 1999a) and accounts for the myriad ways in which we all learn (Tomlinson, 
1999b). Undifferentiated instruction and curriculum, conversely, may stifle student enthusiasm 
for learning and ultimately for achieving to the fullest (Kohn, as interviewed by O-Neil & Tell, 
1999). Sizer (1999) similarly points out that a “rigid system” of schooling will ultimately fail 
those students whom it does not accommodate (1999, p.l). “A one-size- fits-all approach to 
classroom teaching is ineffective for most students and harmful to some,” suggest Tomlinson and 
Kalbfleisch (1998, p.l) in their analysis of brain research, because “to learn, students must 
experience appropriate levels of challenge” (p. 3). As Tomlinson put it earlier, “There simply is 
no single learning template” for all students (1995, p.l) 

The Differentiated Instruction subscale developed for the SCA attempts to measure the 
degree to which school faculty adapt their classroom teaching, grouping, and assessment 
practices in order to meet the needs of their various students. Items were constructed by AEL 
staff with close attention to the literature cited above. 
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Expectations for Student Performance 

School staffs expectations for student academic performance play a powerful role in how 
students actually perform. Teachers’ expectations for students inform how they treat students. 

For instance, teachers holding depressed expectations for certain students may then treat them 
differently than other students perceived to be more capable. Such differential treatment, very 
different than the differential instruction described above, results in fewer opportunities to learn 
challenging material, less time to answer questions or complete assignments, and less frequent 
encouragement and praise (deMarrais & LeCompte, 1999; Lumsden, 1997; McLeod, 1987; 
Willis, 1981). Over time, students’ performance conforms to the expectations of teachers 
(Tauber, 1998), thereby confirming teachers’ original expectations. In addition, teachers are in 
positions of power relative to students, making their expectations all the more influential. 

Wilson and Martinussen (1999) show dramatically how teacher expectations based on 
students’ socioeconomic status and prior achievement significantly shape the final grades study 
participants accorded their students. Ogbu (1983) likewise illustrates how important teacher 
expectations are to students’ academic involvement and, ultimately, to their achievement. 

Expectations for student performance are often shaped by stereotypical assessments 
based on race/ethnicity, socioeconomic status, gender, family structure, language, immigrant 
status, religion, transience, sexual orientation, and other contextually significant social 
characteristics (Bourdieu & Passeron, 1997; deMarrais & LeCompte, 1999; McLeod, 1987; 
Ogbu, 1983; Paley, 1979; Riordan, 1997; Willis, 1981). Hence, teachers sometimes may 
anticipate that, for instance, white middle-class boys will perform better academically than 
working-class Latinas (Schwartz, 2001b). This is not to blame teachers for holding differential 
expectations; rather, such expectations are endemic to our stratified society (c.fi, Rose, 1990; 
Takaki, 1987). Nonetheless, American education also seeks to nurture meaningful democratic 
involvement through equal opportunity to all citizens, and in this regard differential expectations 
based on social and economic characteristics run counter to such ideals. 

The Expectations for Student Performance subscale evaluates the degree to which 
teachers expect that their students are capable of mastering material presented to them this 
academic year. It also assesses the level at which teachers believe their students will perform vis- 
a-vis their peers. Items were developed by AEL staff following a review of the literature on the 
impact of teacher expectations on student performance described above. 

Time for Planning 

School improvement efforts may have little chance of success if faculty lack fundamental 
structural support for their implementation. Among the most important of such conditions is the 
provision of adequate time to allow staff to plan, implement, experiment with, and evaluate their 
improvement initiatives (Howley & Brown, 2001; Howley-Rowe, 1999; Raywid, 1993). 
“Insufficient time to plan for implementing [reform] is a common barrier to implementation and 
a frequent concern of teachers,” reports Desimone (2000, p. 12) in her analysis of schools 
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instituting comprehensive school reform. Teachers are better equipped to develop professionally 
if they have time during their work day to reflect, collaborate, and focus on their own learning. 
Such opportunities, moreover, are fundamental to the development of schools as professional 
learning communities (Abdal-Haqq, 1996; Lashway, 1998). Conversely, lack of time to plan and 
implement contributes to teacher turnover (Adelman, Haslem, & Pringle, 1996). 

An adequate allotment of time for reform to be learned about and practiced, 
implemented, institutionalized, assessed, and reflected upon is crucial (Adelman & Walking- 
Eagle, 1997). Some researchers have even argued that time is so important to the success of any 
school improvement undertaking because change proceeds according to standard development 
phases; without time, reform has no chance to develop (Hord, Rutherford, Huling-Austin, & 

Hall, 1987). 

Sufficient time for planning is therefore an important structural resource to which 
teachers require access if reform is to have the opportunity to become institutionalized. For this 
reason, Time for Planning subscale items were developed by AEL staff to evaluate the extent to 
which faculty are provided enough time for within-grade and across-grade planning and for 
appropriate professional development. 

In Sum 

School capacity is an often-used phrase in discussions of educational reform and 
improvement. However, very few researchers have attempted to define and operationalize school 
capacity for improvement (although, see Newmann, King, & Youngs, 2001). Rather, school 
capacity is a vague, albeit appealing, reference to some ephemeral quality predisposing schools 
to successful change. 

AEL staff have attempted to define and operationalize the concept of school capacity 
through the development of the SCA. Nonetheless, we were also interested in testing our 
definition empirically. Thus, a pilot test of the instrument was conducted during the summer of 
2002. 



The purpose of the pilot test of AEL’s SCA was to begin an exploration of the 
instrument’s subscales. AEL staff wanted to discover the correlations between items intended to 
constitute distinct subscales and assess discrete concepts, and to delete items not highly 
correlated with others in their respective subscales. In other words, AEL staff sought data 
reduction, as the 99-item instrument is cumbersome. Staff also were interested in the degree to 
which subscales were reliable. In sum, an exploratory analysis of the SCA’s statistical properties 
was wanted. 

The audience for this report includes AEL staff working with low-performing schools. As 
the SCA is developed further, it may prove to be a useful diagnostic tool as professional 
developers begin their work in ailing school districts or schools. It is anticipated that the 
refinement of the SCA will render it more amenable to use in AEL’s current and future research 
and development efforts. 
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Other audiences include the Department of Education’s Office of Educational Research 
and Improvement, the schools and districts with which AEL is working, and other entities 
undertaking external assistance projects. Evaluators and researchers seeking measures of school 
capacity may also find this report of some use. 
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METHODS 



The pilot version of AEL’s SCA was administered to school staff in two districts in the 
summer of 2002. Both districts have histories of social, economic, and political difficulties, and 
schools in both have had several decades of depressed student achievement. District 
administrative staff agreed that AEL would administer the SCA as part of its research and 
development efforts there, and that data from the initial administration would serve as baseline 
data for a longitudinal study of the effects of AEL’s work. 

Five hundred copies of the instrument were mailed to the superintendent of a southern 
Virginia school division (school districts are called divisions in Virginia) collaborating with AEL 
in the early summer. Each survey was placed in a brown, sealable envelope; upon completing 
their surveys, each teacher then had the option of returning their survey to the brown envelope 
and sealing it in order to protect their confidentiality and anonymity. Surveys were bundled by 
school, so that each was provided enough for all staff. The superintendent was then responsible 
for disseminating the surveys to each school. 

Once teachers had completed their surveys, sealed their envelopes, and returned them to 
their principals, principals were directed to mail the surveys to AEL. Surveys from 10 schools 
and a total of 427 respondents were returned to AEL by mid-July 2002. 

In addition, the SCA was administered to teams from a West Virginia school district 
attending an AEL summer institute on project-based learning in July 2002. Approximately 40 
copies of the instrument were distributed by an AEL evaluator to teams from 10 schools at the 
close of the institute’s first day. Participants were asked to complete the SCA that evening and 
return it to AEL staff the following morning. Twenty-six respondents completed and returned the 
SCA. 



Returned and completed surveys were scanned into an SPSS data file by a trained 
research assistant. Data were then cleaned by AEL staff, and the West Virginia and Virginia data 
files were merged for pilot test analyses in August, 2002. 

Data analyses for the purpose of reducing the number of items on the SCA were 
conducted in August. Descriptive statistics were calculated. More importantly for data reduction, 
however, AEL staff calculated Pearson Product Moment correlations and Cronbach’s internal 
consistency reliability coefficients. Items within subscales that proved to have low correlations 
with other subscale items were eliminated to form the proposed revised field test version of the 
instrument. In general, items with a preponderance of correlations below .30 were eliminated 
from the pilot test version of the SCA. 

Because so many items contribute to the Equitable Practice subscale, staff chose to 
conduct exploratory factor analysis (Garson, 2002) of this subscale alone. Factor analysis 
revealed two distinct factors within the subscale; these were named, and then correlations for 
items in each of the two new subscales were calculated. As with the other SCA subscales, those 
items showing little correlation with their subscale counterparts were not recommended for 
inclusion in the proposed field test version of the instrument. 
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FINDINGS 



Descriptive Statistics 

A total of 453 school staff completed AEL’s pilot version of the SCA. These respondents 
hailed from 20 schools in two districts with which AEL has been collaborating. However, the 
majority (94.3%) of respondents were from the Virginia school division; only 5.7 were from the 
West Virginia district. The number and percentage of respondents from each school are 
presented in Table 1. The number and percentage of respondents by school level are then 
presented in Table 2. 



Table 1 

Frequency and Percent of Respondents by School 



School 


N 


Percent 


VA 1 


38 


8.4 


VA 2 


25 


5.5 


VA 3 


40 


8.8 


VA 4 


59 


13.0 


VA 5 


92 


20.3 


VA 6 


29 


6.4 


VA 7 


53 


11.7 


VA 8 


26 


5.7 


VA9 


33 


7.3 


VA 10 


32 


7.1 


WV 1 


2 


0.4 


WV 2 


5 


1.1 


WV 3 


2 


0.4 


WV 4 


1 


0.2 


WV 5 


5 


1.1 


WV 6 


3 


0.7 


WV 7 


1 


0.2 


WV 8 


3 


0.7 


WV 9 


1 


0.2 


WV 10 


3 


0.7 



Table 2 

Frequency and Percent of Respondents by School Level 



School level 


N 


Percent 


Elementary 


235 


51.9 


Middle 


112 


24.7 


High 


106 


23.4 
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As shown in the previous tables, the percent of respondents from each school ranged 
from a low of 0.2% of the total sample in three West Virginia schools to a high of 20.3% of the 
total sample in one Virginia school. Number of respondents ranged from a low of one respondent 
to a high of 92. In terms of school level, slightly more than half (51 .9%) of respondents to the 
SCA worked in elementary schools. Approximately a quarter each of remaining respondents 
hailed from middle or high schools (24.7% and 23.4%, respectively). 

Table 3 presents descriptive statistics for subscales. These include the number of 
respondents, minimum and maximum subscale mean scores (calculated by dividing subscale 
item values by the number of subscale items), the subscale mean scores, standard deviations, a 
measure of skewness, and Cronbach’s alpha coefficient assessing internal consistency reliability. 
(Descriptive statistics for each item are presented in Appendix B.) 

Overall mean scores for each subscale ranged from a low of 2.32 (SD 0.59) for the 
Deprivatized Practice scale to a high of 2.94 (SD 0.50) for the Differentiated Instruction 
subscale. The Equitable Practice subscale was shown to be highly reliable, with a reliability 
coefficient of .97. The remaining subscale scores also possessed adequate internal consistency 
reliability, although the Time for Planning subscale was somewhat less reliable in this pilot test. 

In terms of skewness, all but the Collective Teacher Efficacy subscale (which appeared in 
this administration to be very symmetrical) tended to have negative skew. The Program 
Coherence, Equitable Practice, Differentiated Instruction, and Expectations for Student 
Performance subscales had significant negative skew, with skewness statistics more than twice 
the standard error for skewness. These findings are not surprising, however, given that the 
schools surveyed are low-performing and collaborating with AEL for the purpose of 
improvement. In fact, negative skew suggests that the pilot version of the SCA is able to identify 
low-performance. 



Table 3 

Descriptive Statistics for SCA Subscales 



Subscale 


N 


Min. 


Max. 


Mean 


SD 


Skew 


Std. 

Err. 


Alpha 


Collective Teacher 
Efficacy 


453 


1.25 


3.91 


2.67 


0.44 


.004 


.115 


.77 


Deprivatized 

Practice 


453 


1.00 


4.00 


2.32 


0.59 


-.093 


.115 


.78 


Program Coherence 


453 


1.00 


3.67 


2.61 


0.41 


-.517 


.115 


.75 


Technical 

Resources 


453 


1.00 


3.71 


2.36 


0.51 


-.032 


.115 


.71 


Equitable Practice 


453 


1.08 


3.97 


2.91 


0.46 


-.691 


.115 


.97 


Differentiated 

Instruction 


451 


1.38 


4.00 


2.94 


0.50 


-.339 


.115 


.86 


Expectations for 

Student 

Performance 


452 


1.10 


4.00 


2.85 


0.51 


-.263 


.115 


.86 


Time for Planning 


453 


1.00 


4.00 


2.46 


0.61 


-.058 


.115 


.69 
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Reliability 



Cronbach’s Alpha was calculated for each subscale as a measure of its internal 
consistency reliability for this administration of AEL’s SCA. In addition, the Cronbach’s Alpha 
was calculated for each item in the hypothetical circumstance that it were deleted from the 
subscale. These statistics are presented in Table 4. 

Table 4 

Cronbach’s Coefficient Alpha by Subscale 



Item Number 


Cronbach’s Alpha if Item Deleted 


Total SCA, Alpha = .97 


Collective Teacher Efficacy, Alpha = . 77 


1 


.74 


2 


.75 


5 


.74 


7 


.75 


9 


.75 


11 


.76 


13 


J7 


15 


.75 


16 


.77 


17 


.75 


19 


.76 


20 


.76 


Deprivatized Practice, Alpha = . 78 


3 


.78 


6 


.76 


8 ^ 


.73 


10 


.73 


12 


_ - 


26 


.75 


43 


.75 


Program Coherence, Alpha = . 75 


22 


.72 


23 


.70 


24 


.69 


27 


.73 


29 


.76 


30 


.79 


33 


.70 


34 


.71 


36 


.76 


37 


.73 




16 



25 



39 


.71 


41 


.71 


Technical Resources, Alpha = .71 


21 


.72 


25 


.62 


28 


.63 


31 


.68 


32 


.69 


38 


439 1 


40 


j66 1 


Equitable Practice, Alpha = .97 


44 


.97 


45 


.96 


48 


.96 


51 


.96 


52 


.96 


54 


.96 


55 


.97 


56 


.96 


57 


.96 


58 


.96 


60 


.96 


61 


.97 


63 


.96 


64 


.96 


66 


.96 


67 


^97 


69 


^ 6 “ 1 


70 


.96 


72 


.96 


73 


.96 


74 


.96 


75 


.96 


76 


.96 


77 


.96 


78 


.96 


81 


.96 


83 


.96 


85 


.96 


88 


.96 


90 


.96 


91 


.96 


92 


.96 


95 


.96 


96 


.96 
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26 



97 


.97 


99 


.96 


Differentiated Instruction, Alpha = .86 


50 


.85 


62 


.85 


68 


.84 


82 


.84 


86 


.84 


87 


.83 


89 


.84 


93 


.89 


Expectations for Student Performance, Alpha = .86 


47 


.87 


49 


.85 


53 


.84 


59 


.84 


65 


.84 


71 


.84 


80 


.86 


84 


.84 


94 


.84 


98 


.84 


Time for Planning, Alpha = .69 


4 


.66 


14 


.64 


18 


.66 


35 


.63 


42 


.61 



Overall, the SCA subscales possess sufficient internal consistency reliability in this 
administration of the instrument. The Equitable Practice subscale is particularly reliable, with a 
Cronbach’s coefficient of .97. Also quite reliable are the Differentiated Instruction and 
Expectations for Student Performance subscales, both with reliability coefficients of .86. Less 
internally consistent is the Time for Planning subscale, with a Cronbach’s Alpha of .69. 

In addition, the total instrument itself yielded very sufficient internal consistency 
reliability for this administration. The Cronbach’s coefficient Alpha for the entire SCA was .97. 

As illustrated in Table 4, the deletion of items does not appear to significantly improve 
the reliability of any of the subscales. Item deletion tends to increase or decrease most subscale 
Alphas by only .09 at the most. 
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Correlations 



Because the Collective Teacher Efficacy and Deprivatized Practice subscales have been 
shown to be valid and reliable in previous research efforts, they were not subject to data 
reduction techniques. However, item correlations were calculated for each remaining subscale. 

Data presented in Table 4 indicate that nine of the Program Coherence items are at least 
moderately correlated with at least two other subscale items. Some correlations were negative, 
which is surprising given that negatively phrased items were reverse coded for analysis. 
Correlations ranged from quite weak, at -.01 1 (Items 36 and 40), to very strong, at .739 (Items 33 
and 34). Many correlations were statistically significant, but this was not the basis upon which 
items were chosen for the edited version of the SCA. Items with correlations .300 and above with 
at least two other subscale items were selected for inclusion in the new field test version of the 
SCA. In terms of the Program Coherence subscale. Items 29, 30, and 36 were eliminated. As 
shown in Table 5, and all following correlation tables, those items retained in the revised field 
test version of the SCA are highlighted in grey. 

Table 5 

Correlations Among Program Coherence Subscale Items 



Item 

No. 


22 


23 


24 


27 


29 


30 


33 


34 


36 


37 


39 


40 


22 


1.000 
























23 


.443** 


1.000 






















24 


.479** 


.618** 


1.000 




















27 


.180** 


.342** 


.312** 


1.000 


















29 


-.033 


.094* 


.054 


-.039 


1.000 
















30 


-.210** 


-.178** 


-.210** 


-.169** 


.387** 


1.000 














33 


.322** 


.413** 


.446 


.216** 


.064 


-.112* 


1.000 












34 


.246** 


.334** 


.438** 


.166** 


.054 


-.122* 


.739** 


1.000 










36 


.012 


.044 


.018 


.062 


.066 


.070 


.039 


.087 


1.000 








37 


.227** 


.279** 


.321** 


.190** 


.070 


-.138** 


.309** 


.232** 


-.179** 


1.000 






39 


.319** 


.367** 


.462** 


.342** 


.090 


-.202** 


.357** 


.346** 


.070 


.243** 


1.000 




40 


.359** 


.455** 


.521** 


.289** 


.056 


-.202** 


.418** 


.366** 


-.011 


.282** 


.477** 


1.000 



"‘"‘Statistically significant at the .01 level. 
* Statistically significant at the .05 level. 



The inter-item correlations for the Technical Resources subscale are shown in Table 6. 
Correlations were all positive and ranged from .049 (Items 21 and 38) to .684 (Items 25 and 28). 
Only three items from this subscale met the correlation strength criterion for inclusion in the 
revised SCA. Items 21, 31, 32, and 38 were eliminated. 
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Table 6 

Correlations Among Technical Resources Subscale Items 



Item 

No. 


21 


25 


28 


31 


32 


38 


40 


21 


1.000 














25 


.080 


1.000 












28 


.081 


.684'"'" 


1.000 










31 


.239’"’" 


. 274 ** 


.280*'" 


1.000 








32 


.067 


. 235 ** 


. 262 ** 


.189** 


1.000 






38 


.049 


. 348 ** 


. 288 ** 


.195** 


.176** 


1.000 




40 


.175'"'" 


. 387 ** 


. 386 ** 


.302** 


.234** 


.219** 


1.000 



’"’"Statistically significant at the .01 level. 



As shown in Table 7, all Differentiated Instruction subscale items are statistically 
significantly correlated at the .01 level. Correlations ranged from .139 (Items 50 and 93) to .682 
(Items 86 and 87) and were all positive. All items except Item 93 were retained in the revised 
SCA. 



Table 7 

Correlations Among Differentiated Instruction Subscale Items 



Item 

No. 


50 


62 


68 


82 


86 


87 


89 


93 


50 


1.000 
















62 


.474** 


1.000 














68 


.572** 


.506** 


1.000 












82 


.470** 


.478** 


.495** 


1.000 










86 


.433** 


.547** 


.511** 


.574** 


1.000 








87 


.513** 


.508** 


.556** 


.606** 


.682** 


1.000 






89 


.485** 


.504** 


.484** 


.541** 


.585** 


.653** 


1.000 




93 


.139** 


.218** 


.197** 


.306** 


.237** 


.274** 


.170** 


1.000 



'"'"Statistically significant at the .01 level. 



All items in the Expectations for Student Performance subscale were correlated at 
statistically significant levels. Correlations ranged from .102 (Items 47 and 94) to .565 (Items 84 
and 94). Item 47 was not selected for inclusion in the field test version of the SCA, however, as 
no correlations achieved the .300 criterion. 




2 o 29 



Table 8 

Correlations Among Expectations for Student Performance Subscale Items 



Item 

No. 


47 


49 


53 


59 


65 


71 


80 


84 


94 


98 


47 


1.000 




















49 


.166** 


1.000 


















53 


.271** 


.482** 


1.000 
















59 


.174** 


.350** 


.496** 


1.000 














65 


.163** 


.422** 


.508** 


.454** 


1.000 












71 


.218** 


.348** 


.496** 


.558** 


.534** 


1.000 










80 


.239** 


.220** 


.305** 


.304** 


.398** 


.399** 


1.000 








84 


.157** 


.514** 


.476** 


.388** 


.428** 


.445** 


.227** 


1.000 






94 


.102* 


.371** 


.423** 


.371** 


.434** 


.421** 


.217** 


.565** 


1.000 




98 


.136** 


.398** 


.504** 


.423** 


.504** 


.547** 


.316** 


.547** 


.489** 


1.000 



** Statistically significant at the .01 level. 
♦Statistically significant at the .05 level. 



Because the Equitable Practice subscale contained so many items, AEL staff suspected 
that it included several factors. For the purposes of data reduction, exploratory factor analysis 
(Garson, 2002) was conducted on the entire SCA, with especial attention given to the way in 
which factor analysis revealed factors within the Equitable Practice subscale in the context of the 
whole instrument. Principal component analysis was used, and the data were rotated via the 
varimax method to limit the correlation of identified factors. Varimax solutions are orthogonal 
and tend to have either large or small loadings of particular variables on each factor. In other 
words, shared variance is reduced. In addition, six factors were forced, as an earlier factor 
analysis had identified a total of 17 factors (this factor analysis was not used for the purposes of 
this pilot study, however, because the sample was not sufficiently balanced in terms of school 
performance). Again, the purpose of this technique at this juncture in the development of the 
SCA is to reduce the number of items on the instrument. 



The factor analysis identified two distinct factors underlying the Equitable Practice items. 
The first explained 14.31% of the SCA’s total variance, and the second, 12.23% of the total 
variance. Items with factor loadings of .500 and above were selected for correlation analysis and 
potential inclusion in the field test version of the instrument. 

The first factor, tentatively entitled the Anti-Discriminatory Teaching subscale, consists 
of items that take a more active stance toward teaching equitably. For example, some of the 
items loading on this factor include Teachers in this school . . . discourage disparaging treatment 
of students based on race/ethnicity, use language that does not demean or exclude, teach 
students to challenge stereotypes, and are aware of their own biases. This factor is made up of 
16 items. 



The second factor, provisionally called the Responsive Pedagogy subscale, is made up of 
items that concern responsiveness to students’ communities, the creation of equitable classroom 
environments, and language and text use. Some items include Teachers in this school . . . Ensure 
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that all students participate in classroom activities, respond to diverse community interests, show 
an interest in learning about diversity, and interact with students in ways that acknowledge 
students ’ speaking style preferences regardless of how similar they are to staff members ’ 
preferences. In all, 9 items constitute this subscale. 

Table 9 presents the correlations between items contributing to the new Anti- 
Discriminatory Teaching subscale. Not surprisingly (because of their basis in the factor analysis), 
all items were statistically significantly correlated at the .01 level, and all correlations were 
positive. Correlations ranged from .278 (Items 55 and 91) to .702 (Items 77 and 90). Because of 
the high correlations between all items, all were retained for inclusion in the field test version of 
the SC A. 




22 



31 



Table 9 

Correlations Among the New Anti-Discriminatory Teaching Subscale Items 
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Table 10 presents correlation coefficients for the Responsive Teaching subscale items. 

All items were positively correlated and were statistically significantly correlated at the .01 level. 
Correlations varied from .367 (Items 45 and 70) to .668 (Items 57 and 73). In addition, all 
correlations met the .300 criterion for inclusion in the revised version of the SCA. 

Table 10 

Correlations Among the New Responsive Teaching Subscale Items 



Item 

No. 


45 


57 


96 


51 


52 


67 


69 


70 


73 


45 


1.000 


















57 


.554""" 


1.000 
















96 


.520""" 


.584""" • 


1.000 














51 


.413** 


.462""" 


.518""" 


1.000 












52 


.498""" 


.484** 


.539""" 


.582""" 


1.000 










67 


.420""" 


.466""" 


.515""" 


.389""* 


.433""" 


1.000 








69 


.504""" 


.540""" 


.535""" 


.493""" 


.478""" 


.561""" 


1.000 






70 


.367""" 


.452""" 


.379""" 


.477""" 


.383""" 


.425** 


.515""* 


1.000 




73 


.498""" 


.668""" 


.530""" 


.462""" 


.486""" 


.428""" 


.566"'"' 


.512"'"' 


1.00 



"""Statistically significant at the .01 level. 



All items in the Time for Planning subscale were statistically significantly correlated, as 
shown in Table 1 1. Correlations ranged from .189 (Items 4 and 14) to .505 (Items 14 and 42) and 
were all positive. However, only Items 14, 35, and 42 met the .300 criterion for inclusion in the 
field test version of the instrument. 



Table 11 

Correlations Among Time for Planning Subscale Items 



Item 

No. 


4 


14 


18 


35 


42 


4 


1.000 










14 


.189'"'" 


1.000 








18 


.450** 


.203** 


1.000 






35 


.290*'" 


.358** 


.210** 


1.000 




42 


.214** 


.505** 


.250** 


.474** 


1.000 



'"'"Statistically significant at the .01 level. 



Correlations between each original subscale were also calculated. As shown in Table 12, 
all subscales were at least minimally correlated. Correlations ranged from relatively weak, with a 
correlation coefficient of .260 for the Equitable Practice and Time for Planning subscales, to very 
strong, with a correlation of .861 for the Equitable Practice and Differentiated Instruction 
subscales. With a correlation of .747, Equitable Practice and Expectations for Student 
Performance were strongly related. Differentiated Instruction and Expectations for Student 
Performance were also highly correlated, with a correlation coefficient of .727. 
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Table 12 

Correlations Among SCA Subscales 
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o 
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Resources 
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.300** 
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* 

VO 

CN 

CO 


.518** 


Program 

Coherence 
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O 

O 
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.493** 


* 

* 
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* 
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OO 


.568** 
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* 
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* 

* 

OO 

CO 


.386** 
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* 
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ON 
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.415** 
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The weak correlation between Equitable Practice and Time for Planning is not surprising; 
intuitively, one would not necessarily expect the two concepts to share much in common. On the 
other hand, the strong relationship between the Equitable Practice, Differentiated Instruction, and 
Expectations for Student Performance subscales is a phenomenon one might anticipate. As 
discussed in the introduction, these practices and attitudes together constitute a stance toward 
pursuing meaningful and equitable student achievement. It is not difficult to imagine teachers 
who hold challenging standards for their students, are committed to providing equitable 
educational opportunities to students, and who additionally differentiate instruction so as to 
ensure that all students are given adequate support to achieve. (It should be noted that equity, in 
this scenario and as related to the development of the SCA, does not mean that all students are 
treated identically. Rather, this conception of equity is one in which students are provided equal 
access to the support they need, regardless of the variation in needs.) 

Also positively and moderately correlated are the Collective Teacher Efficacy and 
Expectations for Student Performance subscales, with a correlation of .610. Again, such a 
correlation is to be expected. Teachers’ sense of their capacity to teach is closely linked to their 
expectations for how students will perform. In other words, teachers who do not have faith in 
their ability to teach students well will likely not expect their students to perform well. 

Program Coherence is correlated, with a correlation coefficient of .568, with Time for 
Planning. Similarly, Technical Resources is related to Time for Planning, with a correlation 
coefficient of .518. These subscales were intended to assess some of the structural conditions 
under which teachers work. It is therefore confirming that these concepts are correlated. 
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CONCLUSIONS AND RECOMMENDATIONS 



Conclusions 

The SCA appears to hold some promise for assessing school capacity for improvement, 
based on the findings from this pilot test. As would be expected given the nature of the sample of 
low-performing schools, item and subscale means were relatively low. In addition, the sample 
evidenced a negative skew on nearly all of the subscales. These findings suggest that the SCA is 
able to identify struggling schools, although it is yet to be seen if the instrument is also capable 
of identifying schools with a great deal of capacity for improvement. 

In terms of reliability, the instrument overall was internally consistent in the 
administration. In addition, most of the subscales possessed sufficient internal consistency 
reliability. These findings lead to the conclusion that the instrument overall and each subscale 
together constitute concepts that can be assessed reliably over time. 

Correlations and exploratory factor analyses indicated the relationship of subscale items 
to one another. Most subscale correlation matrices confirmed the moderate to strong relationship 
between subscale items. Those items with weak correlations with other subscale items were not 
retained for inclusion in the revised version of the SCA, thereby achieving one of the primary 
purposes of the pilot test — to reduce the number of SCA items. 

Exploratory factor analyses differentiated further the Equitable Practice subscale into the 
Anti-Discriminatory Teaching and Responsive Pedagogy subscales. Items in each are moderately 
to highly correlated. 

Correlations between the subscales were, in general, moderate to very strong. Subscales 
assessing structural conditions were highly correlated with one another, whereas the subscales 
evaluating practice and attitudinal stances were very highly to moderately correlated. This 
suggests that the overall instrument assesses the two sorts of subscales effectively, and that the 
subscales are interrelated without sacrificing their distinct measures. 

Recommendations 

Based on the conclusions above, it is recommended that the SCA undergo further 
development. First, AEL staff should revise the pilot test version of the instrument so that it 
reflects the findings from this study. The Collective Teacher Efficacy and Deprivatized Practice 
subscales should be retained. Items not correlating with other subscale items should be deleted. 
Also, the Equitable Practice subscale should be divided into the two subscales discerned by the 
exploratory factor analysis. As a result, the revised version of the SCA would include the 
subscales and items presented in Table 13, as well as the previously validated Collective Teacher 
Efficacy and Deprivatized Practice subscales. 

Further development would additionally entail the conduct of a field test, in which a 
larger, more differentiated sample would be used. In other words, ideally, the SCA would be 
administered to an equal number of respondents from low-, moderate-, and high-performing 
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schools so as to test its ability to discriminate between the various levels of school performance. 
A larger, more diverse sample would also allow factor analysis for the purpose of more 
rigorously and empirically defining the subscales. 

Table 13 

Revised SCA Items and Subscales 



Program Coherence = 9 items 


Supplemental programs like Title I are carefully designed to complement classroom teaching. 


We routinely evaluate the utility of efforts to improve our school. 


Professional development activities are aligned to achieve improvement goals at this school. 


We practice organized abandonment of programs that are not doing what we expected them to do. 


Curriculum, instruction, and materials are well coordinated across the different grade levels at this school. 


Curriculum, instruction, and materials are well coordinated within the different grade levels at this school. 


Most professional development topics are offered once in the school and not followed up. 


Professional development in this school is supported by other initiatives for improving the school. 


We choose innovations selectively. 


Technical Resources = 6 items 


Teachers have adequate equipment in their classroom (e.g., computers, maps, lab materials) to teach to their 
objectives. 


I have all the equipment and materials I need to teach my students well. 


The texts and instructional materials students have allow teachers to teach to their objectives. 


Regular time is set aside at this school for teachers to coordinate curriculum, instruction, or materials within the 
different grade levels at this school. 


Teachers in this school have sufficient time available for professional development that is appropriate to our school’s 
goals. 


Regular time is set aside at this school for teachers to coordinate curriculum, instruction, or materials across the 
different grade levels. 


Anti-Discriminatory Teaching = 16 items 
Teachers in this school . . . 


Discourage disparaging treatment of students based on their racial/ethnic background. 


Teach students to be culturally sensitive. 


Encourage students to value each others’ differences. 


Use language that does not demean or exclude any students. 


Ensure that students participate in classroom activities that are sensitive to their racial/ethnic background. 


Discourage disparaging comments about students based on their racial/ethnic background. 


Assign materials that do not promote stereotypes. 


Acknowledge the contributions of various racial or ethnic groups. 


Discourage disparaging treatment of students based on their socioeconomic status. 


Teach students to challenge stereotypes. 


Express the same level of respect for the abilities of all students. 


Offer praise to students in an even-handed, equitable manner. 


Discourage disparaging comments about students based on their socioeconomic status. 


Are aware of their own biases. 


Encourage students to acknowledge each others’ strengths. 


Assign reading material written by or about women. 


Responsive Pedagogy = 9 items 
Teachers in this school . . . 


Provide learning experiences that make subject matter meaningful for students from diverse backgrounds. 


Ensure that students participate in classroom activities that are respectful of their socioeconomic status. 


Use culturally sensitive assessment strategies. 


Respond to diverse community interests. 


Provide culturally relevant and respectful learning environment for students. 
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Learn about their students by participating in their communities. 


Connect curriculum to students’ home and community experiences. 


Interact with students in ways that acknowledge students’ speaking style preferences regardless of how similar they 
are to staff members’ preferences. 


Show an interest in learning about diversity. 


Differentiated Instruction = 7 items 

Teachers in this school . . . 


Use a variety of assessment techniques to accommodate diverse learners. 


Use classroom grouping methods that are flexible. 


Provide varied learning environments to accommodate diverse learners. 


Understand how students differ in their approaches to learning. 


Use various classroom grouping methods. 


Differentiate instruction to promote all students’ achievement. 


Modify individual students’ assignments to groups based on regular assessments of individual students’ performance. 


Expectations for Student Performance = 9 items 
Teachers in this school . . . 


Expect most students in this school to perform above the national average in terms of academic achievement. 


Have high expectations for all students. 


Believe that most students in this school are able to master basic skills. 


Expect almost all students to perform at or above grade level by the end of this year. 


Believe that children here have what it takes to leam this year’s material. 


Do not expect many students to master basic skills at each grade level. 


Believe that most students at this school will perform at about the national average in terms of academic 
achievement. 


Believe that most students in this school are capable of performing at about the national average. 


Think that their students will work hard this year. 



It is likewise recommended that the field test include validity and further reliability tests. 
Test-retest reliability could be conducted, for instance. In terms of validity, a concurrent validity 
test might be conducted using an instrument similar to the SCA. Construct validity could be 
tested via factor analysis and comparison of known groups in the sample (that is, low- and high- 
performing schools would be compared). 

Finally, AEL staff might want to consider renaming the instrument so that it better 
reflects the construct it seeks to measure. The instrument is intended to assess school capacity for 
improvement; therefore, the title should reflect a focus on improvement as well. 
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AEL School Capacity Assessment Questionnaire (SCA) 



Your School: 



Identification Number 



Your School District: 



Directions: Please fill in the six digits of your “Identification Number” 
on the grid to the right. This number ensures anonymity and is easy to generate. 
It consists of the last two digits of your Social Security Number (SSN), the 
last two digits of your home phone number, and the two digits for the day of the 
month on which you were bom (01-31). 

Read each item and then rate the extent to which it is true for your school. 

Using a scale of 1- 4 for your ratings, with a “1” indicating “Strongly Disagree” 
and a “4” indicating “Strongly Agree,” completely fill in the bubble that 
represents how well you believe your school performs. 



Like this*. 



Not like this: 






SSN# 


Home # 


Day ' 


Bom 


® 


® 


® 


® 


® 


® 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 
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© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 



Strongly Disagree 


Disagree 


Agree 


Strongly Agree 


© 


© 


© 


© 



Teachers here are confident they will be able to motivate their students. 


© 


© 


© 


© 


If a child doesn’t want to learn, teachers here give up. 


© 


© 


© 


© 


I have often visited a peer’s school to observe and discuss his/her teaching/leaming 
situation. 


© 


© 


© 


© 


I have to give up planning time because of non-academic activities at this school. 


© 


© 


© 


© 


Teachers in this school are able to get through to difficult students. 


© 


© 


© 


© 


Other than formal evaluation, my supervisor(s) regularly observe(s) my work in 
school and give(s) me meaningful feedback. 


© 


© 


© 


© 


These students come to school ready to learn. 


© 


© 


© 


© 


I have often been visited by a peer from another classroom to observe and discuss 
my teaching/learning situation. 


© 


© 


© 


© 


Teachers in this school do not have the skills to deal with student disciplinary problems. 


© 


© 


© 


© 


Two or more colleagues in the building regularly observe my work in school and 
give me meaningful feedback. 


© 


© 


© 


© 


Teachers here don’t have the skills needed to produce meaningful student learning. 


© 


© 


© 


© 


I have often been visited by a peer from another school to observe and discuss my 
teaching/learning situation. 


© 


© 


© 


© 


Home life provides so many advantages students here are bound to learn. 


© 


© 


© 


© 


Regular time is set aside at this school for teachers to coordinate curriculum, instruction, 
or materials within the different grade levels at this school 


© 


© 


© 


© 


Students here just aren’t motivated to learn. 


© 


© 


© 


© 


Learning is more difficult at this school because students are worried about their safety. 


© 


© 


© 


© 
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Strongly Disagree 



Disagree 



Agree 



Strongly Agree 



© © © 




© 






17. Teachers in this school really believe every child can learn. 


© 


© 


© 


© 


18. I have to give up instructional time because of non-academic events at this school. 


© 


© 


© 


© 


19. Drugs and alcohol abuse in the community make learning difficult for students here. 


© 


© 


© 


© 


20. The opportunities in this community ensure that these students will learn. 


© 


© 


© 


© 


21. The tests students take reflect teachers’ objectives. 


© 


© 


© 


© 


22. Supplemental programs like Title I are carefully designed to complement 
classroom teaching. 


© 


© 


© 


© 


23. We routinely evaluate the utility of efforts to improve our school. 


© 


© 


© 


© 


24. Professional development activities are aligned to achieve improvement goals at 
this school. 


© 


© 


© 


© 


25. Teachers have adequate equipment in their classroom (e.g., computers, maps, 
lab materials) to teach to their objectives. 


© 


© 


© 


© 


26. I receive informal, meaningful feedback on my performance from my peers. 


© 


© 


© 


© 


27. We practice organized abandonment of programs that are not doing what we 
expected them to do. 


© 


© 


© 


© 


28. I have all the equipment and materials I need to teach my students well. 


© 


© 


© 


© 


29. We have so many different programs in this school that I can’t keep track of them all. 


© 


© 


© 


© 


30. We have many new programs in our school. 


© 


© 


© 


© 


31. There is adequate space in classrooms to allow teachers to teach to their objectives. 


© 


© 


© 


© 


32. Teachers have access at school to journals and professional resource materials. 


© 


© 


© 


© 


33. Curriculum, instruction, and materials are well coordinated across the different 
grade levels at this school. 


© 


© 


© 


© 


34. Curriculum, instruction, and materials are well coordinated within the different grade 
levels at this school. 


© 


© 


© 


© 


35. Teachers in this school have sufficient time available for professional development 
that is appropriate to our school’s goals. © 


© 


© 


© 




36. When we add a new program, we often eliminate another one. 


© 


© 


© 


© 


37. Most professional development topics are offered once in the school and not 
followed up. 


© 


© 


© 


© 


38. Classroom equipment usually does not function adequately. 


© 


© 


© 


© 


39. Professional development in this school is supported by other initiatives for 
improving the school. 


© 


© 


© 


© 


40. The texts and instructional materials students have allow teachers to teach to 
their objectives. 


© 


© 


© 


© 


41. We choose innovations selectively. 


© 


© 


© 


© 


42. Regular time is set aside at this school for teachers to coordinate curriculum, 
instruction, or materials across the different grade levels. 


© 


© 


© 


© 


43. I have often visited a peer’s classroom to observe and discuss his/her 
teaching/learning situation. 


© 


© 


© 


© 
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Strongly Disagree 

© 



Disagree 

© 



Agree 

© 



Strongly Agree 



© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 
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© 
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© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 



For Questions 44-99 , the professional staff at this school: 

44. Understand the community(ies) in which their students live. 

45. Provide learning experiences that make subject matter meaningful for students 
from diverse backgrounds. 

46. Expect students to be culturally sensitive. 

47. Tend to underestimate students’ performance. 

48. Show respect for students’ differences. 

49. Expect most students in this school to perform above the national average in terms 
of academic achievement. 

50. Use a variety of assessment techniques to accommodate diverse learners. 

51. Ensure that students participate in classroom activities that are respectful of 
their socioeconomic status. 

52. Use culturally sensitive assessment strategies. 

53. Have high expectations for all students. 

54. Respond to diverse community interests. 

55. Discourage disparaging treatment of students based on their racial/ethnic 
background. 

56. Teach students to be culturally sensitive. 

57. Provide culturally relevant and respectful learning environment for students. 

58. Encourage students to value each others’ differences. 

59. Believe that most students in this school are able to master basic skills. 

60. Seek to prevent their biases from hindering student learning. 

61. Teach students to recognize stereotypes. 

62. Use classroom grouping methods that are flexible. 

63. Teach students to challenge discrimination. 

64. Ensure all students’ participation in instructional activities. 

65. Expect almost all students to perform at or above grade level by the end of this year. 

66. Assign reading material written by or about members of various racial/ethnic groups. 

67. Learn about their students by participating in their communities. 

68. Provide varied learning environments to accommodate diverse learners. 

69. Connect curriculum to students’ home and community experiences. 

70. Interact with students in ways that acknowledge students’ speaking style preferences 
regardless of how similar they are to staff members’ preferences. 

71. Believe that children here have what it takes to learn this year’s material. 

72. Help students apply their learning to their home and community. 

73. Show an interest in learning about diversity. 



Strongly Disagree 


Disagree 


Agree 


Strongly Agree 


o 


© 


© 


© 



74. Use multicultural examples when explaining content. 


© 


© 


© 


© 


75. Use language that does not demean or exclude any students. 


© 


© 


© 


© 


76. Ensure that students participate in classroom activities that are sensitive to their 
racial/ethnic background. 


© 


© 


© 


© 


77. Discourage disparaging comments about students based on their racial/ethnic 
background. 


© 


© 


© 


© 


78. Assign materials that do not promote stereotypes. 


© 


© 


© 


© 


79. Acknowledge the contributions of various racial or ethnic groups. 


© 


© 


© 


© 


80. Do not expect many students to master basic skills at each grade level. 


© 


© 


© 


© 


81. Discourage disparaging treatment of students based on their socioeconomic status. 


© 


© 


© 


© 


82. Understand how students differ in their approaches to learning. 


© 


© 


© 


© 


83. Teach students to challenge stereotypes. 


© 


© 


© 


© 


84. Believe that most students at this school will perform at about the national average 
in terms of academic achievement. 


© 


© 


© 


© 


85. Express the same level of respect for the abilities of all students. 


© 


© 


© 


© 


86. Use various classroom grouping methods. 


© 


© 


© 


© 


87. Differentiate instruction to promote all students’ achievement. 


© 


© 


© 


© 


88. Offer praise to students in an even-handed, equitable manner. 


© 


© 


© 


© 


89. Modify individual students’ assignments to groups based on regular assessments 
of individual students’ performance. 


© 


© 


© 


© 


90. Discourage disparaging comments about students based on their socioeconomic status. 


© 


© 


© 


© 


91. Are aware of their own biases. 


© 


© 


© 


© 


92. Provide learning experiences that make subject matter interesting for students 
from diverse backgrounds. 


© 


© 


© 


© 


93. Do not believe that students have different learning styles. 


© 


© 


© 


© 


94. Believe that most students in this school are capable of performing at about the 
natural average. 


© 


© 


© 


© 


95. Encourage students to acknowledge each others’ strengths. 


© 


© 


© 


© 


96. Respond to diverse community interests. 


© 


© 


© 


© 


97. Assign reading material written by or about women. 


© 


© 


© 


© 


98. Think that their students will work hard this year. 


© 


© 


© 


© 


99. Design instructional activities that reflect local community interests. 


© 


© 


© 


© 
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Collective Teacher Efficacy 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Teachers here are 
confident they will be able 
to motivate their students. 


450 


2.86 


3.00 


3 


.827 


-.480 


.115 


1 


4 


If a child doesn’t want to 
learn, teachers here give 
up. 


452 


3.23 


3.00 


3 


.829 


-1.061 


.115 


1 


4 


Teachers in this school are 
able to get through to 
difficult students. 


445 


2.61 


3.00 


3 


.819 


-.344 


.116 


1 


4 


These students come to 
school ready to learn. 


443 


2.00 


2.00 


2 


.861 


.269 


.116 


1 


4 


Teachers in this school do 
not have the skills to deal 
with student disciplinary 
problems. 


452 


2.98 


3.00 


3 


.892 


-.582 


.115 


1 


4 


Teachers here don’t have 
the skills needed to 
produce meaningful 
student learning. 


448 


3.35 


3.00 


4 


.747 


-1.113 


.115 


1 


4 


Home life provides so 
many advantages students 
here are bound to learn. 


449 


1.62 


1.00 


1 


.842 


1.253 


.115 


1 


4 


Students here just aren’t 
motivated to learn. 


449 


2.60 


3.00 


3 


.898 


-.229 


.115 


1 


4 


Learning is more difficult 
at this school because 
students are worried about 
their safety. 


450 


3.34 


3.00 


4 


.702 


-.939 


.115 


1 


4 


Teachers in this school 
really believe every child 
can learn. 


451 


3.05 


3.00 


3 


.823 


-.619 


.115 


1 


4 


Drugs and alcohol abuse 
in the community make 
learning difficult for 
students here. 


447 


2.31 


2.00 


2 


.899 


.273 


.115 


1 


4 


The opportunities in this 
community ensure that 
these students will learn. 


442 


2.02 


2.00 


2 


.781 


.403 


.116 


1 


4 




Deprivatized Practice 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


I have often visited a 
peer’s school to observe 
and discuss his/her 
teaching/leaming 
situation. 


441 


2.15 


2.00 


1 


.969 


.243 


.116 


1 


4 


Other than formal 
evaluation, my 
supervisor(s) regularly 
observe(s) my work in 
school and give(s) me 
meaningful feedback. 


447 


2.66 


3.00 


3 


.981 


-.340 


.115 


1 


4 


I have often been visited 
by a peer from another 
classroom to observe and 
discuss my 
te aching/leaming 
situation. 


440 


2.27 


2.00 


3 


.939 


.084 


.116 


1 


4 


Two or more colleagues in 
the building regularly 
observe my work in 
school and give me 
meaningful feedback. 


445 


2.36 


2.00 


3 


.910 


-.041 


.116 


1 


4 


I have often been visited 
by a peer from another 
school to observe and 
discuss my 
teaching/leaming 
situation. 


434 


1.79 


2.00 


2 


.779 


.699 


.117 


1 


4 


I receive informal, 
meaningful feedback on 
my performance from my 
peers. 


447 


2.59 


3.00 


3 


.790 


-.479 


.115 


1 


4 


I have often visited a 
peer’s classroom to 
observe and discuss 
his/her teaching/leaming 
situation. 


436 


2.35 


2.00 


3 


.868 


-.072 


.117 


1 


4 




Program Coherence 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Supplemental programs 
like Title I are carefully 
designed to complement 
classroom teaching. 


433 


2.69 


3.00 


3 


.868 


-.520 


.117 


1 


4 


We routinely evaluate the 
utility of efforts to 
improve our school. 


442 


2.66 


3.00 


3 


.831 


-.325 


.116 


1 


4 


Professional development 
activities are aligned to 
achieve improvement 
goals at this school. 


445 


2.68 


3.00 


3 


.847 


-.463 


.116 


1 


4 


We practice organized 
abandonment of programs 
that are not doing what we 
expected them to do. 


433 


2.30 


2.00 


2 


.797 


.054 


.117 


1 


4 


We have so many 
different programs in this 
school that I can’t keep 
track of them all. 


443 


3.02 


3.00 


3 


.775 


-.624 


.116 


1 


4 


We have many new 
programs in our school. 


442 


2.82 


3.00 


3 


.735 


-.192 


.116 


1 


4 


Curriculum, instruction, 
and materials are well 
coordinated across the 
different grade levels at 
this school. 


448 


2.47 


3.00 


3 


.827 


-.341 


.115 


1 


4 


Curriculum, instruction, 
and materials are well 
coordinated within the 
different grade levels at 
this school. 


449 


2.65 


3.00 


3 


.776 


-.662 


.115 


1 


4 


When we add a new 
program, we often 
eliminate another one. 


426 


2.52 


3.00 


3 


.723 


-.170 


.118 


1 


4 


Most professional 
development topics are 
offered once in the school 
and not followed up. 


441 


2.42 


2.00 


2 


.765 


.036 


.116 


1 


4 


Professional development 
in this school is supported 
by other initiatives for 
improving the school. 


442 


2.54 


3.00 


3 


.725 


-.491 


.116 


1 


4 


We choose innovations 
selectively. 


426 


2.55 


3.00 


3 


.709 


-.565 


.118 


1 


4 




Technical Resources 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


The tests students take 
reflect teachers’ 
objectives. 


446 


2.96 


3.00 


3 


.757 


-.681 


.116 


1 


4 


Teachers have adequate 
equipment in their 
classroom (e.g., 
computers, maps, lab 
materials) to teach to their 
objectives. 


452 


1.86 


2.00 


1 


.881 


.588 


.115 


1 


4 


I have all the equipment 
and materials I need to 
teach my students well. 


447 


1.80 


2.00 


1 


.874 


.732 


.115 


1 


4 


There is adequate space in 
classrooms to allow 
teachers to teach to their 
objectives. 


445 


2.54 


3.00 


3 


.878 


-.388 


.116 


1 


4 


Teachers have access at 
school to journals and 
professional resource 
materials. 


449 


2.63 


3.00 


3 


.838 


-.504 


.115 


1 


4 


Classroom equipment 
usually does not function 
adequately. 


445 


2.30 


2.00 


3 


.868 


-.012 


.116 


1 


4 


The texts and instructional 
materials students have 
allow teachers to teach to 
their objectives. 


446 


2.43 


3.00 


3 


.834 


-.332 


.116 


1 


4 





Equitable Practice 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Understand the 
community(ies) in which 
their students live. 


452 


3.02 


3.00 


3 


.769 


-.688 


.115 


1 


4 


Provide learning 
experiences that make 
subject matter meaningful 
for students from diverse 
backgrounds. 


451 


2.82 


3.00 


3 


.726 


-.595 


.115 


1 


4 


Show respect for students’ 
differences. 


448 


3.04 


3.00 


3 


.686 


-.640 


.115 


1 


4 


Ensure that students 
participate in classroom 
activities that are 
respectful of their 
socioeconomic status. 


440 


2.89 


3.00 


3 


.665 


-.801 


.116 


1 


4 


Use culturally sensitive 
assessment strategies. 


438 


2.70 


3.00 


3 


.690 


-.530 


.117 


1 


4 


Respond to diverse 
community interests. 


444 


2.80 


3.00 


3 


.693 


-.533 


.116 


1 


4 


Discourage disparaging 
treatment of students 
based on their 
racial/ethnic background. 


441 


3.05 


3.00 


3 


.792 


-.866 


.116 


1 


4 


Teach students to be 
culturally sensitive. 


439 


2.89 


3.00 


3 


.705 


-.630 


.117 


1 


4 


Provide culturally relevant 
and respectful learning 
environment for students. 


445 


2.97 


3.00 


3 


.660 


-.678 


.116 


1 


4 


Encourage students to 
value each others’ 
differences. 


446 


3.11 


3.00 


3 


.683 


-.694 


.116 


1 


4 


Seek to prevent their 
biases from hindering 
student learning. 


441 


3.00 


3.00 


3 


.645 


-.458 


.116 


1 


4 


Teach students to 
recognize stereotypes. 


442 


2.75 


3.00 


3 


.694 


-.479 


.116 


1 


4 


Teach students to 
challenge discrimination. 


435 


2.78 


3.00 


3 


.749 


-.416 


.117 


1 


4 


Ensure all students’ 
participation in 
instructional activities. 


443 


3.09 


3.00 


3 


.670 


-.650 


.116 


1 


4 


Assign reading material 
written by or about 
members of various 
racial/ethnic groups. 


438 


2.88 


3.00 


3 


.730 


-.489 


.117 


1 


4 


Leam about their students 
by participating in their 
communities. 


439 


2.53 


3.00 


3 


.790 


-.150 


.117 


1 


4 





Equitable Practice, cont 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Connect curriculum to 
students’ home and 
community experiences. 


438 


2.69 


3.00 


3 


.723 


-.326 


.117 


1 


4 


Interact with students in 
ways that acknowledge 
students’ speaking style 
preferences regardless of 
how similar they are to 
staff members’ 
preferences. 


426 


2.79 


3.00 


3 


.697 


-.655 


.118 


1 


4 


Help students apply their 
learning to their home and 
community. 


446 


2.93 


3.00 


3 


.670 


-.641 


.116 


1 


4 


Show an interest in 
learning about diversity. 


446 


2.96 


3.00 


3 


.652 


-.501 


.116 


1 


4 


Use multicultural 
examples when explaining 
content. 


442 


2.80 


3.00 


3 


.725 


-.470 


.116 


1 


4 


Use language that does not 
demean or exclude any 
students. 


445 


3.04 


3.00 


3 


.701 


-.689 


.116 


1 


4 


Ensure that students 
participate in classroom 
activities that are sensitive 
to their racial/ethnic 
background. 


438 


2.87 


3.00 


3 


.678 


-.591 


.117 


1 


4 


Discourage disparaging 
comments about students 
based on their 
racial/ethnic background. 


444 


3.11 


3.00 


3 


.708 


-.844 


.116 


1 


4 


Assign materials that do 
not promote stereotypes. 


440 


3.08 


3.00 


3 


.593 


-.482 


.116 


1 


4 


Acknowledge the 
contributions of various 
racial or ethnic groups. 


440 


3.14 


3.00 


3 


.644 


-.700 


.116 


1 


4 


Discourage disparaging 
treatment of students 
based on their 
socioeconomic status. 


434 


2.97 


3.00 


3 


.770 


-.765 


.117 


1 


4 


Teach students to 
challenge stereotypes. 


437 


2.83 


3.00 


3 


.705 


-.490 


.117 


1 


4 


Express the same level of 
respect for the abilities of 
all students. 


440 


2.99 


3.00 


3 


.713 


-.515 


.116 


1 


4 


Offer praise to students in 
an even-handed, equitable 
manner. 


443 


3.13 


3.00 


3 


.642 


-.486 


.116 


1 


4 




Equitable Practice, cont 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Discourage disparaging 
comments about students 
based on their 
socioeconomic status. 


438 


3.06 


3.00 


3 


.697 


-.732 


.117 


1 


4 


Are aware of their own 
biases. 


433 


2.75 


3.00 


3 


.705 


-.567 


.117 


1 


4 


Provide learning 
experiences that make 
subject matter interesting 
for students from diverse 
backgrounds. 


439 


2.92 


3.00 


3 


.720 


-.503 


.117 


1 


4 


Encourage students to 
acknowledge each others’ 
strengths. 


441 


3.05 


3.00 


3 


.655 


-.641 


.116 


1 


4 


Respond to diverse 
community interests. 


436 


2.84 


3.00 


3 


.674 


-.574 


.117 


1 


4 


Assign reading material 
written by or about 
women. 


432 


2.89 


3.00 


3 


.678 


-.718 


.117 


1 


4 


Design instructional 
activities that reflect local 
community interests. 


438 


2.72 


3.00 


3 


.726 


-.496 


.117 


1 


4 




Differentiated Instruction 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Use a variety of 
assessment techniques to 
accommodate diverse 
learners. 


445 


2.74 


3.00 


3 


.752 


-.436 


.116 


1 


4 


Use classroom grouping 
methods that are flexible. 


442 


2.98 


3.00 


3 


.629 


-.584 


.116 


1 


4 


Provide varied learning 
environments to 
accommodate diverse 
learners. 


440 


2.80 


3.00 


3 


.713 


-.367 


.116 


1 


4 


Understand how students 
differ in their approaches 
to learning. 


443 


3.04 


3.00 


3 


.663 


-.656 


.116 


1 


4 


Use various classroom 
grouping methods. 


444 


2.99 


3.00 


3 


.644 


-.500 


.116 


1 


4 


Differentiate instruction to 
promote all students’ 
achievement. 


439 


2.95 


3.00 


3 


.671 


-.448 


.117 


1 


4 


Modify individual 
students’ assignments to 
groups based on regular 
assessments of individual 
students’ performance. 


442 


2.88 


3.00 


3 


.706 


-.723 


.116 


1 


4 


Do not believe that 
students have different 
learning styles. 


442 


3.14 


3.00 


3 


.783 


-.674 


.116 


1 


4 




Expectations for Student Performance 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


Tend to underestimate 
students’ performance. 


443 


2.70 


3.00 


3 


.789 


-.191 


.116 


1 


4 


Expect most students in 
this school to perform 
above the national average 
in terms of academic 
achievement. 


447 


2.50 


3.00 


3 


.831 


-.105 


.115 


1 


4 


Have high expectations 
for all students. 


447 


3.01 


3.00 


3 


.825 


-.553 


.115 


1 


4 


Believe that most students 
in this school are able to 
master basic skills. 


448 


3.02 


3.00 


3 


.703 


-.695 


.115 


1 


4 


Expect almost all students 
to perform at or above 
grade level by the end of 
this year. 


439 


2.99 


3.00 


3 


.731 


-.473 


.117 


1 


4 


Believe that children here 
have what it takes to learn 
this year’s material. 


447 


2.91 


3.00 


3 


.731 


-.546 


.115 


1 


4 


Do not expect many 
students to master basic 
skills at each grade level. 


445 


3.07 


3.00 


3 


.773 


-.466 


.116 


1 


4 


Believe that most students 
at this school will perform 
at about the national 
average in terms of 
academic achievement. 


442 


2.60 


3.00 


3 


.794 


-.208 


.116 


1 


4 


Believe that most students 
in this school are capable 
of performing at about the 
national average. 


440 


2.78 


3.00 


3 


.771 


-.580 


.116 


1 


4 








Think that their students 
will work hard this year. 


446 


2.92 


3.00 


3 


.771 


-.545 


.116 


1 


4 




Time for Planning 



Item 


N 


Mean 


Median 


Mode 


SD 


Skewness 


Std. Error 
of 

Skewness 


Min. 


Max. 


I have to give up planning 
time because of non- 
academic activities at this 
school. 


444 


2.61 


3.00 


3 


.985 


-.153 


.116 


1 


4 


Regular time is set aside at 
this school for teachers to 
coordinate curriculum, 
instruction, or materials 
within the different grade 
levels at this school. 


450 


2.38 


2.00 


3 


1.009 


.017 


.115 


1 


4 


I have to give up 
instructional time because 
of non-academic events at 
this school. 


442 


2.77 


3.00 


3 


.856 


-.340 


.116 


1 


4 


Teachers in this school 
have sufficient time 
available for professional 
development that is 
appropriate to our school’s 
goals. 


451 


2.32 


2.00 


2 


.838 


-.004 


.115 


1 


4 


Regular time is set aside at 
this school for teachers to 
coordinate curriculum, 
instruction, or materials 
across the different grade 
levels. 


448 


2.24 


2.00 


2 


.832 


.044 


.115 


1 


4 
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Completed Evaluation Standards Checklist 
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Checklist for Applying the Standards 



To interpret the information provided on this form, the reader needs to refer to the full text of the standards as they appear in Joint 
Committee on Standards for Educational Evaluation, The Program Evaluation Standards (1994), Thousand Oaks, CA, Sage. 

The Standards were consulted and used as indicated in the table below (check as appropriate): 



Descriptor 

U 1 Stakeholder Identification 


The Standard was 
addressed 

X 


The Standard was 
partially addressed 


The Standard was 
not addressed 


The Standard was 
not applicable 


U2 Evaluator Credibility 


X 








U3 Information Scope and Selection 


X 








U4 Values Identification 


X 








U5 Report Clarity 


X 








U6 Report Timeliness and Dissemination 


X 








U7 Evaluation Impact 








X 


FI Practical Procedures 


X 








F2 Political Viability 








X 


F3 Cost Effectiveness 


X 








PI Service Orientation 


X 








P2 Formal Agreements 


X 








P3 Rights of Human Subjects 


X 








P4 Human Interactions 


X 








P5 Complete and Fair Assessment 


X 








P 6 Disclosure of Findings 


X 








P7 Conflict of Interest 


X 








P8 Fiscal Responsibility 


X 








A1 Program Documentation 


X 








A2 Context Analysis 


X 








A3 Described Purposes and Procedures 


X 








A4 Defensible Information Sources 


X 








A5 Valid Information 


X 








A6 Reliable Information 


X 








A7 Systematic Information 


X 








A8 Analysis of Quantitative Information 


X 








A9 Analysis of Qualitative Information 








X 


A 10 Justified Conclusions 


X 








A1 1 Impartial Reporting 


X 








A 1 2 Metaevaluation 


X 









The Program Evaluation Standards (1994, Sage) guided the development of this (check one): 

request for evaluation plan/design/proposal 

evaluation plan/design/proposal 

evaluation contract 

evaluation report 

X other: pilot test report 

Name Caitlin Howlev Date S/22/02_ 

o 

(signature) 

Position or Title R & E Specialist 

Agency ^EL 

Address 

Relation to Document Co~author 

(e.g., author of document, evaluation team leader, external auditor, internal auditor) 
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